Skip to content

Add Gemma-4-31B-IT NVFP4 gsm8k accuracy coverage (1xB200) - #39

Open
stecasta wants to merge 1 commit into
vllm-project:mainfrom
stecasta:stecasta/blackwell-nvfp4-small-b200
Open

Add Gemma-4-31B-IT NVFP4 gsm8k accuracy coverage (1xB200)#39
stecasta wants to merge 1 commit into
vllm-project:mainfrom
stecasta:stecasta/blackwell-nvfp4-small-b200

Conversation

@stecasta

@stecasta stecasta commented Jul 21, 2026

Copy link
Copy Markdown

Summary

  • Add nightly gsm8k accuracy recipe for nvidia/Gemma-4-31B-IT-NVFP4 on 1xB200 (TP=1)
  • 2.75M dl/mo; narrowed to Gemma per review (Devstral / Qwen3.6 split into separate PRs)
  • Reference gsm8k (vLLM CI gsm8k harness, 5-shot / 1319q): 0.702

AI-assisted (Claude Code).

Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
@stecasta
stecasta force-pushed the stecasta/blackwell-nvfp4-small-b200 branch from 5d4f57b to 9e59d1f Compare July 27, 2026 09:34
@stecasta stecasta changed the title Add Blackwell NVFP4/FP8 gsm8k accuracy coverage (1xB200) Add Gemma-4-31B-IT NVFP4 gsm8k accuracy coverage (1xB200) Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant